With the development of multimedia technology, interest in realistic media is increasing recently.
Among such technologies, a multi-view video means a set of images obtained by using two or more cameras. Unlike conventional single-view video, multi-view video can generate a three-dimensional dynamic image by capturing one scene through multiple cameras.
The multi-view video can provide users with stereoscopic feeling through free viewpoint and wide screen. Further, a depth map can be extracted from a multi-view image by a method such as stereo matching to generate a three-dimensional stereoscopic image.
Recently, various multi-view video application fields have been studied, and research on freeview TV (FTV), 3-D TV, surveillance, immersive teleconferencing, etc. has been actively conducted.
In order to acquire a multi-view video, a multi-view camera array is constructed by arranging a multi-view camera in a predetermined form.
There are various types of multi-view camera arrays. Among such arrays, a parallel array and an arc array are mainly used, and such arrays can be composed of one-dimensional or two-dimensional arrays. For each multi-view camera array, each camera is placed at a regular distance and angle from the neighboring cameras. In practice, various types of multi-view camera array can be constructed considering the number of cameras, scene, and purpose.
However, the multi-view video has an inevitable geometric error due to errors that occur when placing the cameras according to the multi-view camera array.
These errors represent geometric errors in the one-dimensional parallel camera array and the one-dimensional arc camera array. In principle, the multi-view camera array is set to keep the array intervals and angles of the cameras constant, but when placing the cameras according to the multi-view camera array, errors occur due to the problem of manually placing the cameras. This error refers to the inconsistency of internal parameters such as camera position, direction, and focal length. Thereby, the geometric error in multi-view video makes it difficult to match multiple images of multi-view video.
This can affect the time and accuracy of the matching between images, that is, the three-dimensional image processing technique based on the correlation-depth map generation, intermediate image generation, etc., thereby affecting the coding efficiency. In addition, there is a problem that it is difficult to obtain a smooth viewpoint change in viewing the generated multi-view video. Therefore, a method for compensating the above-described geometric error is required.
In the case of a stereo camera system, image rectification can be used to solve the above described problem. The image rectification method is a method in which all the epipolar lines of two images are paralleled such that the vertical mismatch is straightened.
When image rectification is performed, two image planes are located on the same plane, and corresponding points of the two images have the same vertical coordinate.
However, there are relatively few studies on the correction of geometric errors in multi-view video and there are not many available algorithms.